Statistical mechanical aspects of joint source-channel coding

– An MN-Gallager Code over Galois fields, q, based on the Dynamical Block Posterior probabilities (DBP) for messages with a given set of autocorrelations is presented with the following main results: a) for a binary symmetric channel the threshold, fc, is extrapolated for infinite messages using the scaling relation for the median convergence time, tmed ∝ 1/(fc− f); b) a degradation in the threshold is observed as the correlations are enhanced; c) for a given set of autocorrelations the performance is enhanced as q is increased; d) the efficiency of the DBP joint source-channel coding is slightly better than the standard gzip compression method; e) for a given entropy, the performance of the DBP algorithm is a function of the decay of the correlation function over large distances. With the rapid growth of information content in today’s wire and wireless communication, there is an increasing demand for efficient transmission systems. A significant gain in the transmission performance can be achieved by the application of the joint source-channel coding technique, which has attracted much attention during the recent past, see for instance [1–6]. Roughly speaking, source coding is mainly a data compression process that aims at removing as much redundancy as possible from the source signal, whereas channel coding is the process of intelligent redundancy insertion so as to be robust against channel noise. These two processes, source coding and channel coding, seem to act in opposition, where the first/second process shrinks/expands the transmitted data. For illustration, assume that our compression shrinks the size of the source signal by a factor 2 and in order to be robust against channel noise we have to expand our compressed file by a factor 4. Hence, the length of the transmitted sequence is only twice the length of the uncompressed source. The source-channel coding theorem of Shannon [7] indicates that if the minimal achievable source coding rate of a given source is below the capacity of the channel, then the source can be reliably transmitted through the channel, assuming an infinite source sequence. This theorem implies that source coding and channel coding can be treated separately without any loss of overall performance, hence they are fundamentally separable. Practically, the source can be first efficiently compressed and then an efficient error correction method can be used.